CamelCamelCamel's Communication with the Amazon Service

Introduction#

The C3's services interact with Amazon's services to provide a seamless user experience. In this lesson, we’ll see how this interaction happens in the following two scenarios:

• C3 service communicating client requests to Amazon.

• C3 service communicating server requests (C3 requests) to Amazon.

In the case of client requests, the C3 service prepares a new request based on queries received in the client's request, sends it to the Amazon service, and waits for the response. After receiving a response from Amazon, the C3 service processes it and sends it to the client.

In the case of server requests, the requests are prepared based on generalized queries to get maximum data from Amazon. Let's elaborate on communication between C3 and Amazon in the following sections.

Search service#

Whenever a search request arrives from the client, the C3 service lets the client wait until it receives up-to-date results from Amazon's search service. The illustration below presents the complete flow of a search request by a client:

We already discussed the search message's format and endpoints between a client and service provider (the C3 service in our case). Let's discuss the format of the message from the C3 service and the endpoint, which communicates with the Amazon service in the upcoming section.

Base URL of Amazon service#

The following base URL is used to send requests to Amazon's services:

The following endpoint is used to perform searching and fetching product operations:

Message format for the search#

The data entities are the same as discussed in C3's service. We'll analyze the request and response format now:

• HTTP method: To search for a product, the HTTP POST method fetches a list of available products based on the information passed in payload from the Amazon service.

• Request format: The request format for the HTTP POST method is defined below as given in the product advertising API's request by Amazon:

Amazon's authentication#

Amazon uses unique headers that are defined below to authenticate each request:

x-amz-date: 20160925T120000Z: A custom date header specifies the request's date and time to prevent third parties from intercepting the request for later submission. A request must reach the AWS server within five minutes of the timestamp defined in the request; otherwise, it’s denied by AWS. Moreover, this header is used to create a signature that’s compared later with a signature generated by the client in the authorization header.

x-amz-target: com.amazon.paapi5.v1.ProductAdvertisingAPIv1.SearchItems: This defines the targeted service’s endpoint to place the request. In our case, it’s `SearchItems`. This value eases the API gateway’s process so that it can send requests to the targeted service, and it is also used to create the signature.

Authorization: The authorization header consists of the following parameters:

• AWS4-HMAC-SHA256 defines an algorithm used for signatures and reads as signature version 4 with the HMAC-SHA256 hash algorithm.

• Credential=AKIAIOSFODNN7EXAMPLE is the secret access key assigned to authorized users.

• SignedHeaders=content-type;host;x-amz-date;x-amz-target defines which headers are used to create the signature.

• Signature=&5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924a6f2b5d7 is the signature value calculated using the hash algorithm with the defined headers.

The need for a signature helps to secure the requests by verifying the requestor's identity with a valid access key. To prevent request tampering during transit, the hash is calculated again using all the headers and verified with the one in the request. If they both do not match, the request is denied.

• Response format: If the request is successful, the client receives the following response:

Product advertising service#

The functionality of the product advertising service is a bit different than a typical service provider. The C3 service sends several requests (server-sent requests) to Amazon's product advertising service to continuously get up-to-date data. These requests are periodic and are sent using the SQS service of Amazon.

C3's product service fills the SQS queue based on the allowed number of requests in Amazon's product service. The SQS queue sends each request to Amazon's product service periodically on behalf of C3's product service. The response (containing information on multiple items) from Amazon is returned to C3's product service, which is saved locally in C3's database. The SQS queue is filled again when the last request is fulfilled. This periodic pattern continues, keeping throttling in mind. The following illustration shows the workflow:

Note: SQS queue is filled with requests (due to rate limiting per day or hour) that are sent to the PA-API periodically. Each request’s response contains multiple items processed and saved in the C3's database. The queue is emptied after a while because the rate limiter allows our requests to pass through because of a lull period after the previous burst. After the queue is emptied, it is filled again with requests, and this process continues. It sends requests in multiples to reduce the burden of C3's product service instead of sending them individually to the SQS queue.

In the following section, let's discuss the format for each request sent to the SQS queue.

Message format for the product advertising service#

To get details about products from the product advertising service, the C3 service will interact with PA-API. We’ll target the GetItems endpoint to get details of the product based on productID (ASIN in the case of Amazon's products). We’ll analyze the request and response formats that interact with PA-API below:

• HTTP method: We’ll use the POST method to fetch results from a server.

• Request format: The request format defined by Amazon for HTTP POST method is defined below:

• Response format: If the request is successful, the following response is sent back to the C3 service:

In functional requirements, we mentioned that clients could add any product to the watch list and get notified when prices drop. Based on the user's requirements, the C3 service analyzes fetched data (presented above in the response). If a product’s price meets the user’s required price, it notifies them.

The communication with PA-API of Amazon allows its customers to fetch data that can be used to advertise the products on different platforms, which increases sales. Although it offers a limited number of requests to each client, you can fetch multiple items (in the thousands) in a single request. C3 uses the same strategy to fetch the maximum number of items in each request to update the local database and to facilitate its users.